Federal  Aviation 
Administration 


DOT /FAA/AM-09/5 
Office  of  Aerospace  Medicine 
Washington,  DC  20591 


An  Experiment  to  Evaluate  Transfer  of 
Low-Cost  Simulator-Based 
Upset-Recovery  Training 

Rodney  O.  Rogers1 
Albert  Boquet1 
Cass  Howell1 
Charles  DeJohn2 

^mbry-Riddle  Aeronautical  University 
Daytona  Beach,  FL  32114 

2Civil  Aerospace  Medical  Institute 
Federal  Aviation  Administration 
Oklahoma  City,  OK  73125 


March  2009 


Final  Report 


OK-09-0434-JAH 


NOTICE 


This  document  is  disseminated  under  the  sponsorship 
of  the  U.S.  Department  of  Transportation  in  the  interest 
of  information  exchange.  The  United  States  Government 
assumes  no  liability  for  the  contents  thereof. 


This  publication  and  all  Office  of  Aerospace  Medicine 
technical  reports  are  available  in  full-text  from  the  Civil 
Aerospace  Medical  Institute’s  publications  Web  site: 
www.faa.gov/library/reports/medical/oamtechreports 


Technical  Report  Documentation  Page 


1.  Report  No.  2.  Government  Accession  No. 

DOT/FAA/AM-09/5 


4.  Title  and  Subtitle 

An  experiment  to  evaluate  transfer  of  low-cost  simulator-based  upset- 
recovery  training 


3.  Recipient's  Catalog  No. 


5.  Report  Date 

March  2009 


6.  Performing  Organization  Code 


7.  Author(s) 

Rogers  RO,'  Boquet  A,'  Howell  C,1  Dejohn  Cf 


9.  Performing  Organization  Name  and  Address 

'Embry-Riddle  Aeronautical  University  2FAA  Civil  Aerospace  Medical  Institute 
Daytona  Beach,  FL  321 14  P.O.  Box  25082 

Oklahoma  City,  OK  73125 


12.  Sponsoring  Agency  name  and  Address 

Office  of  Aerospace  Medicine 
Federal  Aviation  Administration 
800  Independence  Ave.,  S.W. 
Washington,  DC  20591 


8.  Performing  Organization  Report  No. 


10.  Work  Unit  No.  (TRAIS) 


1 1 .  Contract  or  Grant  No. 


13.  Type  of  Report  and  Period  Covered 


14.  Sponsoring  Agency  Code 


16.  Abstract 


Many  air  transport  training  programs  provide  simulator-based  upset-recovery  instruction  for  company  pilots. 
However,  apparently  no  prior  research  exists  to  demonstrate  that  such  training  transfers  to  an  airplane  in  flight. 
We  report  on  two-phase  FAA-funded  research  experiment  to  evaluate  upset-recovery  training  transfer. 

In  two  separate  training/ testing  evolutions  involving  two  different  general  aviation  aircraft,  participant  pilots 
were  trained  using  low-cost  desktop  flight  simulation,  then  subjected  to  serious  in-flight  upsets  in  an  aerobatic 
airplane.  Their  performance  in  upset-recovery  maneuvering  was  compared  with  the  performance  of  control 
group  pilots  who  received  no  upset-recovery  training.  Data  collected  during  both  flight  testing  periods  suggest 
that  simulator-based  training  improves  a  pilot’s  ability  to  recover  an  airplane  from  an  upset.  However,  in  the 
most  important  measure  of  upset  maneuvering  skills — minimizing  altitude  loss — trained  pilots  fell  well  short  of 
the  performance  routinely  achieveable  by  pilots  experienced  in  all- attitude  manuevering. 

We  summarize  prior  related  research,  describe  the  experiments,  present  and  analyze  data  collected  during 
both  flight  testing  periods,  and  advance  recommendations  for  future  upset  maneuvering  training.  Although  we 
conducted  flight  testing  in  a  general  aviation  airplane,  our  research  has  important  implications  for  heavy  aircraft 
upset-recovery  trainers. 


17.  Keywords 

Airplane  Upset-Recovery  Training,  Low-Cost 
Flight  Simulation,  Training  Transfer 


19.  Security  Classif.  (of  this  report) 

Unclassified 


Form  DOT  F  1700.7  (8-72) 


18.  Distribution  Statement 

Document  is  available  to  the  public  through  the 
Defense  Technical  Information  Center,  Ft.  Belvoir,  VA 
22060;  and  the  National  Technical  Information 
Service,  Springfield,  VA  22161 


20.  Security  Classif.  (of  this  page)  21.  No.  of  Pages 

Unclassified  1 8 


Reproduction  of  completed  page  authorized 


CONTENTS 


1 .  Prior  Research . 1 

1 . 1  Calspan-Related  Research . 1 

1 .2  Centrifuge-Based  Flight  Simulation . 1 

1.3  Fiuman  Factors  Considerations  in  Upset-recovery  Training . 2 

1 .4  Low-Cost  Simulation . 2 

2.  The  Research  Experiment . 2 

3.  Phase  One  Flight  Testing . 3 

3.1  Upset  Attitudes  and  Dependent  Variables . 3 

3.2  Statistical  Analysis . 3 

3.3.  Interpretation . 3 

3.4  Potential  for  Improved  Training . 5 

4.  Phase  Two  Testing . 5 

4.1  Data  Collection,  Upset  Attitudes,  and  Dependent  Variables . 5 

4.2  Statistical  Analysis . 6 

4.3.  Effect  of  Improved  Training . 7 

4.4.  Possible  Effect  of  Reduced  Sample  Size  in  the  Nose-low  Inverted  Upset  Results . 8 

4.5.  Limitations  of  Ground-Based  Simulator  Training . 10 

5.  Implications  for  Transport  Type  Airplanes . 11 

6.  Follow-On  Research . 11 

7.  Endnotes . 12 

8.  List  of  Sources . 13 


iii 


An  Experiment  to  Evaluate  Transfer  of  Low-Cost 
Simulator-Based  Upset-Recovery  Training 


An  upset  occurs  when  an  airplane  enters  an  unex¬ 
pected  attitude  that  threatens  loss  of  control  (LOC)  and 
subsequent  ground  impact.  For  the  years  1998  -  2007 
inclusive,  LOC  was  the  leading  cause  of  hull  losses  and 
passenger  fatalities  in  worldwide  air  transport  operations, 
causing  almost  25%  of  all  crashes  and  nearly  40%  of  all 
fatalities.1  During  the  years  1991  -  2000,  statistics  for 
general  aviation  (GA)  accidents  in  the  United  States  are 
similar,  while  in  Australia  LOC  accounted  for  an  even 
greater  proportion  of  GA  accidents  and  fatalities.2 

Since  LOC  threatens  both  passengers  and  flight  crews, 
as  well  as  potential  victims  on  the  ground,  many  air 
transport  training  programs  contain  a  module  instructing 
pilots  how  to  recover  an  airplane  from  an  upset.  However, 
the  effectiveness  of  such  training  has  not  been  demon¬ 
strated.  We  will  report  on  a  two-phase  Federal  Aviation 
Administration-funded  research  experiment  designed  to 
evaluate  transfer  of  upset-recovery  training  conducted 
using  low-cost  flight  simulation.  We  assess  training  ef¬ 
fectiveness  by  means  of  in-flight  upset-recovery  testing 
in  a  general  aviation  airplane.  In  what  follows,  we: 

1 .  Discuss  prior  research  related  to  our  experiment. 

2.  Describe  the  experiment. 

3.  Summarize  previously  reported  results  of  Phase 
One  flight  testing. 

4.  Present  and  interpret  results  of  Phase  Two  flight 
testing. 

5 .  Construe  the  relevance  of  our  results  to  air  transport 
upset-recovery  training. 

6.  Explain  plans  for  future  related  research. 

1.  PRIOR  RESEARCH3 

We  have  found  only  a  few  research  articles  related  to 
the  transfer  of  simulator-based  upset-recovery  training. 
Several  reports  result  from  research  at  the  Calspan  In-Flight 
Upset  Recovery  Training  Program  in  Roswell,  N.M.4  A 
second  set  of  articles  focuses  on  centrifuge-based  flight 
simulators  manufactured  by  the  Environmental  Tecton¬ 
ics  Corporation.  A  third  group  discusses  human  factors 
considerations  in  upset-recovery  training.  Finally,  we  are 
aware  of  just  one  article  related  to  training  transfer  when 
upset  maneuvering  is  taught  using  low-cost  simulation. 


1.1  Calspan- Related  Research 

Calspan  provides  in-flight  simulator-based  upset- 
recovery  training  in  a  variable  stability  Learj  et  25  modified 
to  simulate  the  control  characteristics  of  an  air  transport 
airplane.  The  Calspan  Lear  can  simulate  various  accident 
scenarios  that  in  the  past  have  resulted  in  air  transport 
upsets  leading  to  uncontrolled  crashes. 

Gawron  used  Calspan’s  Learj  et  to  test  five  groups 
of  airline  pilots  with  varying  degrees  of  upset-recovery 
training  and/or  aerobatics  experience  on  a  series  of  eight 
upsets,  hypothesizing  that  pilots  with  more  training 
and/or  experience  would  outperform  those  with  less. 
However,  she  found  no  significant  difference  among  the 
performances  of  the  five  groups.5 

Kochan  used  the  Calspan  Lear  to  examine  the  roles 
of  domain  knowledge  and  judgment  in  upset-recovery 
proficiency.  Domain  knowledge  is  specific  knowledge 
about  upset-recovery  procedures.  Judgment  is  the  abil¬ 
ity  to  analyze  and  learn  from  an  in-flight  upset-recovery 
experience.  She  tested  four  groups  of  participants  on  a 
series  of  three  in-flight  upsets.  Statistical  analyses  revealed 
that  judgment  was  a  significant  factor  in  successful  upset 
recovery,  especially  when  a  pilot  has  low  domain  knowl¬ 
edge,  i.e.,  when  a  pilot  is  not  trained  to  proficiency  in 
upset  recovery.6 

Kochan  and  Priest  studied  the  effect  of  upset-recovery 
training  in  the  Lear.  They  measured  pre-  and  post- training 
pilot  performance  in  recovering  from  a  series  of  upsets. 
Statistical  analysis  indicated  “a  strong  positive  influence 
of  the  [Calspan  program]  on  a  pilot’s  ability  to  respond 
to  an  in-flight  upset.”7 

Kochan,  Breiter,  Hilscher,  and  Priest  surveyed  reten¬ 
tion  of  knowledge  in  Calspan-trained  pilots.  Although 
participants  in  retrospect  “rated  their  ability  to  recover 
from  loss-of-control  situations  as  being  greatly  improved 
by  the  training,  ’’most  were  unable  to  recall  various  specific 
details  about  upset-recovery  maneuvering  taught  during 
their  training.8 

1.2  Centrifuge-Based  Flight  Simulation 

The  Calspan  Learjet  in-flight  simulator  allows  pilots 
to  experience  upset  maneuvering  G  forces  that  very 
few  ground-based  flight  simulators  can  replicate.  The 
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Environmental  Tectonics  Corporation  (ETC)  ofSouth- 
ampton,  Pennsylvania,  manufactures  centrifuge-based 
flight  simulators  capable  of  generating  continuous  G 
forces.  Such  simulators  bring  to  ground-based  upset- 
recovery  training  a  degree  of  realism  unachievable  even 
in  Level-  D  simulators.  Three  ETC  proprietary  technical 
reports  (available  from  Dick  Leland  at  dletc@aol.com) 
detail  the  capabilities  of  the  company’s  current  genera¬ 
tion  of  centrifugal  simulators. 910,11  One  drawback  of 
such  simulators,  however,  is  that  “if  a  pilot  moves  his 
head  while  under  G  in  a  centrifuge,  strong  feelings  of 
disorientation  (the  Coriolis  illusion)  result  because  of 
the  small  rotation  radius  needed  to  create  the  G  forces 
artificially.”12  In  a  related  article  on  motion-based  flight 
simulation,  Szczepanski  and  Leland  argue  that  “simulator 
data  analysis  suggests  that  motion  cueing  is  necessary 
when  training  ab  initio  pilots  or  pilots  who  have  lim¬ 
ited  or  no  experience  in  the  particular  flying  task  that 
is  being  trained.”13 

Since  upset  maneuvering  involves  generating  high  G 
forces  than  many  pilots  have  never  experienced,  it  seems 
reasonable  to  wonder  if  a  centrifuge-based  simulator 
might  not  be  more  effective  that  low-cost  or  Level-D 
flight  simulators  for  upset-recovery  training.  We  return 
to  this  subject  later. 

1.3  Human  Factors  Considerations  in  Upset- 
recovery  Training 

A  number  of  papers  examine  the  “surprise”  or  “startle” 
factor  in  aviation,  an  effect  that  can  hinder  a  pilot’s  ability 
to  respond  appropriately  to  an  emergency  situation  such  as 
an  upset.  Kochan,  Breiter,  andjentsch  (2004)  found  pilots 
often  miss  cues  that  might  lead  to  avoiding  an  emergency 
that  later  arrives  as  a  surprise.14  In  a  follow-on  paper,  the 
same  researchers  develop  “a  conceptual  framework  for 
the  study  of  unexpected  events  in  aviation.”15  Kochan, 
Priest,  and  Moskal  use  a  model  for  the  “cognitive  process 
of  surprise”16  to  study  “how  an  unexpected  event  can  es¬ 
calate  to  a  loss-of-control  situation.”  They  conclude  that 
in-flight  (as  opposed  to  ground-based)  simulator  training 
may  be  necessary  to  teach  pilots  to  deal  adequately  with 
their  perceptual  biases  in  processing  information  during 
a  surprise  upset.1718  In  a  related  paper,  Kochan  argues 
that  a  pilot’s  response  to  unexpected  events  can  be  im¬ 
proved  through  cognitive  flexibility  training  (to  discourage 
formulaic  and  encourage  flexible  responses  to  surprise 
events),  adaptive  expertise  training  (to  reinforce  modified 
or  new  responses  to  surprise  based  on  responses  learned 
in  previous  expert  training),  and  metacognitive  training 
(to  teach  pilots  how  to  evaluate  their  mental  processes 
in  responding  to  surprise).19 


1.4  Low-Cost  Simulation 

Roessingh  (2005)  studied  training  transfer  from  low- 
fidelity  ground-based  flight  simulators  to  control  of  an 
actual  airplane  during  aerobatic  flight.20Two  experimental 
groups  received  ground-based  instruction  in  aerobatic 
maneuvering  using  desktop  flight  simulators.  The  simu¬ 
lator  syllabus  was  the  same  for  both  groups,  but  one 
experimental  group’s  simulator  training  was  enhanced 
with  a  more  “realistic  layout  of  stick,  rudder  pedals,  and 
throttle.”  Then  the  two  experimental  groups  and  a  control 
group  received  five  hours  of  in-flight  aerobatic  training. 
Data  collected  during  subsequent  testing  revealed  no 
significant  difference  in  the  aerobatic  maneuvering  of 
experimental  and  control  group  pilots. 

2.  THE  RESEARCH  EXPERIMENT 

Our  research  seeks  to  determine  if  upset-recovery  train¬ 
ing  in  low-cost  flight  simulators  develops  flying  skills  that 
improve  a  pilot’s  ability  to  recover  from  a  serious  upset 
in  a  real  airplane.  We  hypothesize  that  pilots  trained  in 
upset  recovery  will  outperform  untrained  pilots.  To  test 
this  hypothesis,  we  train  a  group  of  participant  pilots  in 
upset-recovery  maneuvering  usingMicrosoft  Flight  Simu¬ 
lator  (MFS),21  then  subject  both  trained  and  untrained 
participants  to  a  series  of  four  upset  situations  in  an  actual 
airplane  and  collect  data  on  their  performance  in  recover¬ 
ing  the  airplane  to  straight- and-level  flight.  Participants 
in  our  experiment  are  student  pilots  at  Embry-Riddle 
Aeronautical  University  (ERAU)  in  Daytona  Beach, 
Florida.  All  participants  hold  a  current  instrument  rating 
and  have  completed  an  academic  course  in  basic  aerody¬ 
namics  for  pilots.  None  has  prior  aerobatic  experience 
or  upset-recovery  training  beyond  that  required  for  FAA 
flight  certificates  and  ratings. 

As  reflected  inTable  1 ,  our  experiment  is  a  2  x  4  repeated 
measures  factorial.  The  first  independent  variable  is  degree 
of  training  and  has  two  levels — trained  and  untrained. 
Trained  participants  receive  ten  hours  of  classroom  upset 
training  and  ten  hours  of  MFS  upset-recovery  training.22 
Untrained  participants — control  group  pilots — receive  no 
classroom  or  simulator  training.  The  second  independent 
variable  is  upset  attitude.  It  has  four  levels  corresponding 
to  the  four  upsets  each  participant  is  subjected  to  during 
flight  testing.  We  categorize  upset  attitudes  as  nose-high 
or  nose-low  and  as  upright  or  inverted.  An  inverted  at¬ 
titude  is  one  where  the  bank  angle  exceeds  90°. 

Flight  testing  subjects  participants  to  upsets  intended  to 
simulate  an  airplane  in  cruise  flight  suddenly  disturbed  by 
an  external  force  such  as  severe  wind  shear  or  very  strong 
wake  turbulence.  Participants  close  their  eyes  while  the 
safety  pilot  induces  an  upset,  then — when  instructed  to 
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Table  1 .  The  2x4  Factorial  Design 


2x4  Factorial 

Upset  Attitude  (Repeated  Measure) 

Nose-high 

Upright 

Nose-low 

Upright 

Nose-high 

Inverted 

Nose-low 

Inverted 

Training 

10  Hours  Classroom  /  Simulator 
(Trained  Group) 

Trained  pilots 

Trained  pilots 

Trained  pilots 

Trained  pilots 

None  (Control  Group) 

Untrained  pilots 

Untrained  pilots 

Untrained  pilots 

Untrained  pilots 

do  so — open  their  eyes  and  attempt  to  bring  the  airplane 
under  control.  If  a  participant  pilot  returns  the  aircraft 
to  straight-and-level  flight  without  verbal  or  physical 
assistance  from  the  safety  pilot,  a  recovery  is  successful; 
otherwise,  it  is  unsuccessful. 

We  define  a  good  upset  recovery  as  one  where  a  pilot 
respects  aircraft  operating  limitations  while  returning  the 
aircraft  to  straight-and-level  flight  with  the  minimum 
possible  loss  of  altitude.  Minimum  altitude  loss  will 
result  from: 

•  Prompt  and  correct  control  and  throttle  inputs  in 
response  to  an  upset  situation. 

•  A  high  roll  rate  toward  an  upright  attitude  to  orient 
the  lift  vector  toward  the  sky. 

•  Use  ofappropriate  G  forces  (unloaded  during  low-speed 
or  inverted  rolls;  high  Gs  in  dive  pullouts  while  avoid¬ 
ing  accelerated  stalls)  during  upset  maneuvering. 

The  dependent  variables  in  our  experiment  are  designed 
to  measure  these  factors.  Upset  attitudes  and  dependent 
variables  differ  slighdy  between  Phase  One  and  PhaseTwo 
of  our  research  because  we  used  different  test  airplanes, 
with  different  instrumentation,  in  each  of  the  two  phases. 
This  situation,  however,  does  not  constitute  a  design 
limitation:  no  data  from  Phase  One  testing  were  used  in 
Phase  Two  analysis. 

3.  PHASE  ONE  FLIGHT  TESTING23 

3.1  Upset  Attitudes  and  Dependent  Variables 

We  conducted  Phase  One  flight  testing  in  a  Beech 
Bonanza  airplane  provided  by  the  Calspan  Flight  Research 
Group.  Table  2  indicates  the  initial  attitudes  and  airspeeds 
associated  with  each  of  the  four  upsets.  Nose-high  upset 
airspeeds  were  set  at  10  KIAS  above  V  ,  the  Bonanza  1 
G  stall  speed.  Nose-low  airspeeds  were  a  maximum  safe 
value  taking  into  consideration  the  Bonanza  never-exceed 
speed  of  196  KIAS.  Data  values  for  dependent  variables 
were  derived  from  a  flight  data  recorder  and  a  video  file 
produced  from  a  camera  focused  on  the  Bonanza’s  instru¬ 
ment  panel.  Table  3  shows  dependent  variables  used  in 
statistical  analysis. 


3.2  Statistical  Analysis 

During  flight  testing,  we  experienced  difficulties  with 
data  collection  hardware  that  resulted  in  lost  or  inaccurate 
data.  In  addition,  we  discarded  data  for  upsets  where  the 
initial  aircraft  attitude  and  airspeed  varied  significantly 
from  target  values  show  in  Table  2.  Finally,  we  omitted 
data  for  14  unsuccessful  upset  recoveries,  six  by  trained 
participants  and  eight  by  control  group  participants. 
As  a  consequence,  although  we  tested  28  trained  and 
30  control  group  participants  during  232  upsets,  we 
produced  reliable  data  for  far  fewer  upsets,  as  shown 
in  Table  4.  Note  that  the  table  reflects  the  number  of 
individual  upsets  with  reliable  data  for  each  upset,  not 
the  number  of  participants  with  complete  data  sets  for 
all  four  upsets. 

Because  unusable  data  resulted  in  too  few  complete  data 
sets,  a  two-group  repeated  measures  Multivariate  Analyses 
of  Covariance  (MANCOVA)  proved  infeasible.  Thus,  we 
decided  to  analyze  each  upset  separately  to  preserve  a  rea¬ 
sonable  sample  size.  For  each  upset  in  order,  we  calculated 
the  Wilks’  Lambda  value.  Since  each  analysis  revealed  a 
significant  multivariate  effect,  we  then  used  univariate 
ANOVAs  employing  the  Bonferroni  adjustment  to  assess 
the  contribution  of  individual  dependent  measures. 
Table  5  records  the  mean  and  standard  deviation  (in 
italics)  for  each  dependent  measure.  Bolding  indicates 
a  significant  effect. 

3.3.  Interpretation 

Statistical  analysis  implies  a  strong  relationship  be¬ 
tween  training  in  low-cost  flight  simulators  and  improved 
control  responses  during  a  serious  upset  situation  in  a 
real  airplane.  Trained  pilots  outperformed  control  group 
pilots  in  six  of  the  nine  dependent  measures,  reflecting 
a  superior  control  of  G  forces  during  unloaded  rolls  and 
dive  pullouts,  an  increased  willingness  to  use  large  roll 
control  inputs,  and  quicker  throttle  responses.  The  result 
was  a  tendency  to  return  the  aircraft  to  straight-and-level 
flight  faster  than  control  group  pilots.  In  three  dependent 
measures,  however — seconds  to  first  roll;  rudder  input; 
and,  most  notably,  altitude  loss — trained  pilots  never 
outperformed  control  group  pilots. 
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Table  2.  Levels  of  the  Upset  Attitude  Independent  Variable  for  Phase  One  Testing 


Upset 

Pitch 

Bank 

Airspeed 

Thrust 

Nose- high  Upright 

6CP  Nose-high 

15°  LdL  Wing  Down 

70KIAS 

Idle 

Nose-low  Upright 

3C P  Nose-low 

75°  Left  Wing  Down 

120KIAS 

Cruise 

Nose-high  Inverted 

6CP  Nose-high 

135°  Right  Wing  Down 

70KIAS 

Idle 

Nose- low  Inverted 

2CP  Nose-low 

135°  Left  Wi  ng  Down 

100KIAS 

Cruise 

Table  3.  Dependent  Variables  Used  in  Phase  One  Statistical  Analysis 

Dependent  Measures 

G  Force  in  Dive  Pullout 

Ratio  of  Available  to  Allowable  G  in  Dive  Pullout 

_ G  Force  Unloading  during  Rolls _ 

Recovery  Altitude  Loss  in  Feetf 
Time  to  First  Throttle  Response  in  Seconds 
Time  to  First  Roll  Response  in  Seconds 
Time  to  Recover  in  Seconds 

Use  of  Ailerons  for  Roll  Authority  in  Degrees  of  Yoke  Rotation 
Use  of  Rudder  for  Roll  Authority  in  Inches  of  Rudder  Pedal  Displacement 

"f  Some  nose-high  upset  recoveries  resulted  in  small  altitude  gains,  i.e.,  in  negative  altitude  losses. 


Table  4.  Group  Size  for  Each  Upset 


Group 

Upset 

Nose-High 

Upright 

Nose-Low 

Upright 

Nose-High 

Inverted 

Nose-Low 

Inverted 

Control 

16 

17 

19 

19 

Experimental 

23 

23 

22 

21 

Total 

39 

40 

41 

40 

Table  5.  Dependent  Measures  Means  and  Standard  Deviations  (Bolding  =  Significant  Effect) 


Upset 

Nose-Low  Upright 

Nose-Hig 

;h  Upright 

Nose-Low  Inverted 

Nose-High  Inverted 

Trained 

Control 

Trained 

Control 

Trained 

Control 

Trained 

Control 

G  Use  in  Dive 

3.13 

2.50 

1.88 

1.36 

3.49 

2.92 

2.56 

2.00 

Recovery 

0.50 

0.38 

0.29 

0.20 

0.59 

0.58 

0.82 

0.57 

Average  G / 

0.79 

0.63 

0.93 

0.91 

0.87 

0.75 

0.84 

0.64 

Target  G 

0.13 

0.09 

0.14 

0.12 

0.15 

0.20 

0.15 

0.16 

Unload  G 

1.37 

1.72 

0.34 

0.23 

1.34 

1.71 

0.52 

0.66 

0.28 

0.34 

0.26 

0.20 

0.52 

0.63 

0.59 

0.81 

Altitude  Loss 

918.26’ 

1001.76’ 

12.17’ 

-119.38’ 

1290.95’ 

1243.68’ 

563.18’ 

538.95’ 

225.70’ 

223.53’ 

188.39’ 

226.41  ’ 

371.82’ 

338.68’ 

481.43’ 

380.63’ 

Seconds  to  First 

2.13 

3.12 

1.91 

4.38 

1.57 

2.74 

2.45 

4.79 

Throttle 

1.10 

2.34 

0.51 

5.25 

.51 

1.52 

7.77 

4.34 

Seconds  to  First 

2.00 

2.18 

2.30 

2.06 

1.76 

2.11 

2.36 

2.58 

Roll 

0.85 

1.01 

1.11 

0.99 

0.94 

1.56 

0.85 

1.68 

Seconds  to 

5.96 

8.18 

11.08 

12.31 

7.43 

8.32 

9.14 

12.00 

Recover 

1.07 

2.21 

2.86 

1.77 

1.69 

1.45 

2.34 

3.27 

Roll  Input 

24.22° 

17.18° 

63.30" 

46.38° 

25.67° 

23.16° 

40.64° 

40.00° 

13.73° 

4.79 ° 

18.16° 

73.77° 

13.54° 

9.60° 

20.44° 

18.46° 

Rudder  Input 

0.18” 
0.12 ” 

.20” 

0.15” 

0.91” 

0.28” 

.75” 

0.58” 

0.33” 

0.24” 

0.23” 

0.20” 

1.05” 

0.58” 

0.79” 

0.76” 
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While  trained  pilots  applied  larger  roll  imputs  and 
controlled  G  forces  better  than  untrained  pilots,  both 
groups  used  relatively  small  control  inputs  in  comparison 
to  those  required  for  optimal  recoveries.  Minimum  altitude 
loss  results  from  high  roll  rates  (achieved  at  low  airspeeds 
by  using  both  aileron  and  rudder)  and  from  the  prompt 
application  of  maximum  available/ allowable  G  during  dive 
pullout.  However,  general  aviation  pilots  are  accustomed 
to  the  small  control  inputs  required  for  straight-and-level 
flight  or  flight  at  small  bank  and  pitch  angles.  Without 
prior  aerobatic  experience,  pilots  find  it  difficult  to  apply 
large  control  inputs  during  a  high-stress  upset  situation.  We 
term  this  tendency  general  aviation  syndrome.  An  equally 
appropriate  name  for  this  hard-to-resist  behavior  would  be 
straight-and-level  syndrome,  as  it  also  appears  to  characterize 
professional  pilots  who  lack  aerobatic  experience — most 
notably  airline  pilots  undergoing  upset- recovery  training.24 
This  behavior  was  exhibited  by  trained  participants  to  only  a 
slightly  lesser  degree  than  untrained  participants.  We  believe 
general  aviation  syndrome,  which  our  training  moderated 
but  failed  to  overcome,  explains  why  the  superior  control 
manipulations  by  trained  pilots  did  not  result  in  smaller 
altitude  losses.  The  magnitude  of  trained  pilot  control 
inputs,  though  statistically  significant  in  comparison  to 
untrained  pilot  control  input,  was  not  close  enough  to 
optimal  to  result  in  an  altitude  loss  difference.  Moreover, 
as  explained  in  Subsection  3.4,  in  some  cases  our  train¬ 
ing  methods  likely  increased  (rather  than  decreased)  the 
altitude  losses  experienced  by  trained  pilots. 

3.4  Potential  for  Improved 
Training 

During  Phase  One  flight  test¬ 
ing,  we  identified  areas  where 
improved  ground  training  might 
result  in  better  pilot  performance 
during  flight  testing.  The  first 
concerns  the  practice  of  leveling 
the  wings  before  dive  pullout,  as 
opposed  to  performing  a  rolling 
pullout.  If  less  than  an  optimal  roll 
rate  is  employed  to  level  the  wings 
before  applying  high  G,  or  if  less 
than  allowable  G  is  applied  dur¬ 
ing  pullout,  a  rolling  pullout  will 
decrease  altitude  loss.  Second,  the 
“standard”  recovery  technique  for 
nose-high  upsets,  which  involves 
inducing  a  steep  bank  angle  to 
allow  the  nose  to  fall  toward  the 
horizon  before  leveling  the  wings, 
is  not  optimal  for  high  thrust-to- 
weight  ratio  airplanes.  In  high 


thrust-to-weight  aerobatic  airplanes,  far  less  altitude  loss 
results  when  a  pilot  rolls  immediately  toward  a  wings-level 
upright  attitude  while  flying  nose-high  at  full  power  “across 
the  top”  of  a  half-loop.  This  recovery  technique  mimics  the 
final  roll-out  portion  of  an  Immelman  aerobatic  maneuver. 
Third,  more  emphasis  is  required  during  training  to  help 
pilots  overcome  general  aviation  syndrome  by  using  large 
control  inputs.  Simulator  training  should  stress  unloading 
fully  during  nose-low  inverted  rolls  or  when  rolling  at  low 
airspeeds  during  nose-high  upsets.  It  must  also  emphasize 
the  use  of  large  rudder  as  well  as  aileron  inputs  to  increase 
roll  rates  at  low  airspeeds.  Finally,  it  must  stress  the  need  to 
apply  high  G  forces  quickly  during  dive  recovery  once  the 
lift  vector  is  pointed  toward  the  sky.  In  Phase  One  training, 
we  were  less  than  perfect  in  all  four  instructional  areas. 

4.  PHASE  TWO  TESTING 

4.1  Data  Collection,  Upset  Attitudes,  and 
Dependent  Variables 

After  repeated  failed  efforts  to  find  a  suitably  instru¬ 
mented  contract  aircraft,  we  decided  to  use  the  Embry- 
Riddle  Super  Decathlon  aerobatic  trainer  for  Phase  Two 
flight  testing.  To  collect  data,  we  installed  a  battery-oper¬ 
ated  video  camera  focused  on  the  Decathlon’s  instrument 
panel.  Ahigh-resolution  palm-size  video  recorder  captured 
the  camera’s  output  and  cockpit  voice  communications. 
Figure  1  presents  a  screen  capture  of  a  video  recorded 
during  flight  testing.  We  also  installed  an  Appareo  AHARS 
data  recorder,  an  inexpensive  battery-operated  GPS-based 


Figure  1 .  Sample  Decathlon  Video  Recorder  Output 
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Table  6.  Levels  of  the  Upset  Attitude  Independent  Variable  for  Phase  Two  Testing 


Upset 

Pitch 

Bank 

Airspeed 

Thrust 

Nose-high  Upright 

60°  Nose -high 

45°  Left  Wing  Down 

65  MPH 

Idle 

Nose-low  Upright 

45°  Nose-low 

70°  Right  Wing  Down 

130  MPH 

Full 

Nose-high  Inverted 

60°  Nose -high 

180°  (Inverted,  Wings  Level) 

65  MPH 

Idle 

Nose-low  Inverted 

20°  Nose-low 

180°  (Inverted,  Wings  Level) 

110  MPH 

Full 

Table  7.  Dependent  Variables  Used  in  Phase  Two  Statistical  Analysis 

_ Dependent  Measure _ 

_ Maximum  G  Force  in  Dive  Pullout _ 

_ Minimum  G  Force  Unloading  during  Rollst _ 

_ Recovery  Altitude  Loss  in  Feet:  Negative  Value  =  Altitude  Gain _ 

_ Time  to  First  Throttle  Response  in  Seconds _ 

_ Time  to  First  Roll  Response  in  Seconds _ 

_ Time  to  Recover  in  Seconds _ 

"("Not  applicable  to  the  nose  low  upright  upset,  since  trained  pilots  were  taught  to  use  rolling 
pullouts  during  dive  recovery. 


Table  8.  Flight  Hour  Data  for  50  Flight-Tested  Participants 


Group 

Group  Size 

Mean  Flight  Hours 

Standard  Deviation 

Control 

26 

160.5 

54.0 

Experimental 

25 

201.2 

85.9 

Combined 

51 

180.5 

73.8 

system  capable  of  recording  aircraft  position,  altitude, 
airspeed,  attitude  (pitch  and  bank),  G  forces  (x,  y,  and 
z),  yaw  angles  ((3),  and  similar  parameters.  However,  this 
unit  proved  unreliable  in  aerobatic  attitudes.  As  a  conse¬ 
quence,  we  were  able  to  obtain  only  G  force  data  from  it. 
Two  factors  prevented  our  installing  a  more  sophisticated 
data  recording  system.  One  was  the  significant  cost.  The 
other  is  a  prohibition  against  invasive  instrumentation  in 
an  ERAU  training  aircraft. 

Because  we  used  the  Decathlon  rather  than  the  Bo¬ 
nanza  for  Phase  Two  testing,  we  modified  upset  initial 
attitudes  and  airspeeds  slightly  from  those  used  in  Phase 
One.  Table  6  reflects  upset  attitudes  and  airspeeds  for 
Phase  Two  flight  testing.  Nose-high  initial  airspeeds 
were  set  12  MPH  above  Vs  for  the  Decathlon,  while 
nose-low  airspeeds  reflect  a  maximum  safe  value  based 
on  the  Decathlon’s  red  line  speed  VNE  of  200  mph.  The 
principal  change  in  attitudes  is  that  we  used  180°  of  bank 
(inverted  wings-level  attitude)  for  both  of  the  inverted 
upsets.  This  change  made  it  easier  for  the  safety  pilot 
to  position  the  airplane  in  the  prescribed  attitudes,  a 
challenging  task  significantly  complicated  by  the  fact 
that  the  Decathlon  rear  cockpit,  where  the  safety  pilot 
sits,  has  no  flight  instruments.  The  change  was  prudent, 
given  the  difficulty  we  experienced  during  Phase  One  in 
positioning  the  Bonanza  accurately. 


Table  7  details  the  six  dependent  measures  we  used  to 
conduct  Phase  Two  statistical  analyses.  Lacking  a  reliable 
flight  data  recorder,  we  were  unable  to  collect  data  on 
rudder  and  aileron  input  that  we  obtained  during  Phase 
One.25  We  consider  the  absence  of  these  data  points 
inconsequential,  because  rudder  and  aileron  input  are 
responsible  for  roll  rate,  and  Phase  One  results  established 
training  transfer  in  using  roll  authority.  In  Phase  Two, 
our  main  objective  was  to  determine  if  improved  training 
would  result  in  significantly  smaller  altitude  losses  for 
trained  pilots,  compared  with  control  group  pilots. 

4.2  Statistical  Analysis 

As  a  result  of  careful  safety  pilot  training,  we  were  able 
to  position  the  Decathlon  accurately  for  upsets  and  to 
collect  complete  data  sets  for  24  of  25  trained  participants 
and  for  25  of 26  control  group  participants.  Average  flight 
time  for  each  group  is  shown  in  Table  8.  Six  trained  pilots 
and  eight  control  group  pilots  experienced  unsuccessful 
recoveries  during  the  nose-low  inverted  upset.  In  every 
case,  the  safety  pilot  took  control  in  dive  pullout  to  avoid 
exceeding  the  Decathlon’s  red  line  speed.  Data  for  these 
upsets  are  not  included  in  our  statistical  analysis. 

To  compare  trained  and  control  group  pilot  performance, 
we  conducted  one-way  MANOVAs  for  each  of  the  four  up¬ 
sets  using  the  dependent  measures  in  Table  7.  The  resulting 
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Table  9.  Multivariate  Wilks’  Lambda  Values  and  Group  Sizes  for  Each  Upset 


Upset 

Nose-Low 

Upright 

Nose-High 

Upright 

Nose-Low 

Inverted 

Nose-High 

Inverted 

Trained  Group  Size 

n=25 

n=25 

n=19 

n=24 

Control  Group  Size 

n=26 

n=26 

n=17 

n=26 

Combined  Group  Size 

n=51 

n=51 

n=36 

o 

II 

c 

Wilks  Lambda  Value 

F  (5,45)  =9.59 

p  =  .0001 

rp  =  .0.52 

F(6,44)  =  4.47 

p  =  .001 

r|2=  .38 

F  (6,29)  =9.11 

p  =  .0001 

rf=  .653 

F  (6,43)  =10.26 

p  =  .0001 

rf=.60 

fable  10.  Dependent  Measures  Means  and  Standard  Deviations  (Bolding  =  Significant  Difference 


Upset 

Nose-Low  Upright 

Nose-Hig 

;h  Upright 

Nose-Low  Inverted 

Nose-High  Inverted 

Trained 

Control 

Trained 

Control 

Trained 

Control 

Trained 

Control 

Altitude  Loss 

565.20 

728.46 

331.20 

340.38 

382.08 

464.62 

948.95 

1069.41 

In  Feet 

75.28 

169.51 

225.56 

184.75 

200.65 

169.59 

167.03 

139.08 

Min  Unload  G 

0.93 

0.00 

-0.47 

-0.43 

0.84 

0.99 

in  Rolls 

Not  Applicable! 

0.14 

0.12 

.28 

.26 

0.15 

0.86 

Max  G  in  Dive 

3.70 

2.90 

2.41 

1.82 

2.34 

2.34 

4.74 

3.98 

Pullout 

0.64 

0.49 

0.90 

0.30 

0.45 

0.45 

0.62 

0.50 

Seconds  to 

3.0 

5.19 

2.12 

3.27 

3.31 

3.31 

2.79 

4.41 

First  Throttle 

1.66 

2.43 

1.62 

2.97 

3.21 

3.21 

1.78 

2.81 

Seconds  to 

1.28 

1.85 

2.28 

3.15 

6.15 

6.15 

1.68 

4.88 

First  Roll 

.46 

.68 

.89 

1.38 

2.98 

2.98 

.67 

3.30 

Seconds  to 

5.40 

7.04 

11.16 

12.88 

15.23 

15.23 

7.11 

7.88 

Recover 

1.38 

1.64 

1.43 

2.98 

2.27 

2.27 

1.29 

.99 

t  Not  applicable  to  nose-low  upright  upset  because  trained  pilots  were  taught  to  use  rolling  pullouts  from  dives. 


Wilks’  Lambda  values  shown  in  Table  9  reflect  a  significant 
difference  between  the  two  groups  in  all  four  upsets.  We 
then  conducted  AN OVAs  with  the  Bonferroni  adjustment 
for  each  of  the  dependent  variables.  Table  10  reports  mean 
and  standard  deviation  data  for  the  post  hoc  analyses.  Bold¬ 
ing  in  Table  10  indicates  a  significant  difference.  Figure  2 
presents  information  from  Table  10  in  graphical  format. 
Table  12,  which  follows  in  Subsection  4.3,  presents  F  values 
associated  with  significant  effects  in  Table  10. 

Two  factors  motivated  our  decisions  to  forego  a  more 
traditional  2x4  mixed-model  analysis.  First,  because  we 
eliminated  data  from  unsuccessfully  recoveries  in  the  nose- 
low  inverted  upset,  a  mixed-model  analysis  would  have 
substantially  reduced  the  sample  size  (cf.  Table  9).  This 
approach  maintains  consistency  with  our  Phase  One  ap¬ 
proach,  where  each  upset  was  analyzed  separately  because 
data  collection  problems,  as  well  as  unsuccessfully  recoveries, 
resulted  in  a  small  number  of  complete  data  sets  (cf.  Table 
4).  Second,  the  nature  of  the  upset  data  themselves  argues 
against  the  direct  comparisons  that  characterize  a  repeated 
measures  MAN OVA.  For  example,  a  nose-high  recovery 
may  lead  to  an  altitude  gain,  whereas  nose-low  recoveries 
invariably  result  in  significant  altitude  losses.  Rather  than 
compare  “apples  to  oranges,”  we  opted  for  a  more  direct  and 
operationally  more  relevant  approach  to  data  analysis. 


4.3.  Effect  of  Improved  Training 

As  in  Phase  One,  Phase  Two  statistical  analysis  confirms 
our  hypothesis  that  low-cost  simulator-based  upset- recovery 
training  improves  pilot  performance  in  recovering  an  airplane 
from  a  serious  upset.  However,  Phase  Two  statistical  results 
reflect  a  far  more  comprehensive  training  transfer  than  we 
detected  in  Phase  One.  Phase  Two-trained  pilots  lost  less 
altitude  than  control  group  pilots  in  all  four  upsets,  and  two 
of  the  four  altitude  differences  were  statistically  significant. 
This  result  stands  in  vivid  contrast  to  Phase  One  testing 
outcomes,  where  trained  pilots  actually  lost  more  altitude  than 
control  group  pilots  in  three  of  four  upsets.26  We  attribute 
this  result  to  improved  participant  training,  as  described  in 
Subsection  3.4  above. 

The  improved  performance  in  altitude  loss  occurred 
because  trained  pilots  initiated  rolls  toward  a  wings-level 
upright  attitude  sooner  and  applied  more  Gs  in  dive  pullouts 
than  untrained  pilots,  both  critical  factors  in  minimizing 
altitude  loss.  In  addition,  trained  pilots  also  applied  throttle 
more  promptly  than  untrained  pilots.  These  differences  in 
turn  resulted  in  a  quicker  return  to  straight-and-level  flight. 
Excluding  altitude  loss,  trained  pilots  were  statistically 
superior  to  control  group  pilots  in  14  of  19  categories,  or 
73.7  %  of  the  time.  Including  altitude  loss,  trained  pilot 
performance  exceeded  untrained  pilot  performance  in  1 6  of 
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Nose  High 
Inverted 


Nose  Low 
Inverted 


Max  G  in  Dive  Pullout 

*  "Trained 
*■  Control 

lit 


Time  to  First  Roll  in  Seconds 


*  M 

Jj 


Nose  Low  Nose  High  Nose  High  Nose  Low 
Upright  Upright  Inverted  Inverted 


Nose  Low  Nose  High  Nose  High  Nose  Low 
Upright  Upright  Inverted  Inverted 


23  categories,  or  69.6%  of  the  time.  This  is  in  opposition 
to  16  out  of  36  categories,  or  44.4%  of  the  time,  in  Phase 
One  testing.  These  contrasts  are  presented  in  Tables  1 1  and 
12,  where  an  “X”  or  an  “F”  value,  respectively,  indicates  a 
significant  effect. 

4.4.  Possible  Effect  of  Reduced  Sample  Size  in  the 
Nose-low  Inverted  Upset  Results 

As  previously  explained,  all  14  unsuccessful  recoveries 
occurred  during  the  nose-low  inverted  upset.  Eliminating 
data  from  these  unsuccessful  upsets  results  in  a  sample  size 
of  36,  as  opposed  to  a  sample  size  of  30  when  unsuccess¬ 
ful  recoveries  are  included.  Moreover,  when  data  from 
the  unsuccessful  recoveries  are  included  in  an  analysis 
of  the  nose-low  inverted  upset,  the  MAN OVA  and  all 
six  univariate  Fs  are  significant.  When  it  is  excluded,  the 


univariate  differences  for  Minimum  Unload  G  in  Rolls  and 
Seconds  to  Recover  are  no  longer  significant. 

In  Table  13,  Ratio-A  (All)  is  the  ratio  of  a  dependent 
measure  control  group  average  to  the  corresponding  trained 
group  average  with  data  from  all  recoveries  included.  Ra- 
tio-S  (Successful)  is  the  same  quotient  computed  for  suc¬ 
cessful  recoveries  only.  Eliminating  data  from  unsuccessful 
recoveries  has  little  effect  on  relative  differences  between  the 
two  groups,  as  reflected  in  the  fact  that  Ratio-A  divided  by 
Ratio-S  is  in  every  case  close  to  1 .0,  and  extremely  close  to 
1 .0  in  the  two  categories  where  loss  of  statistical  significance 
occurs  when  unsuccessful  recoveries  are  eliminated.  We 
believe  the  information  in  Table  1 3  suggests  that  the  loss 
of  significance  in  Unload  G  and  Time  to  Recover  differences 
when  unsuccessful  recoveries  are  eliminated  may  be  due 
primarily  to  the  resulting  reduced  sample  size. 


Table  11.  Phase  One  Flight  Testing  Significant  Effects  (X)  as  Determined  by 
Univariate  Analysis  (Rogers  et  al.  [2007],  p.  1 1 ) 


Dependent 

Measure 

Upset 

Nose-Low 

Upright 

Nose-High 

Upright 

Nose-Low 

Inverted 

Nose-High 

Inverted 

G  Use  in  Dive  Recovery 

X 

X 

X 

X 

Average  G  /  Target  G 

X 

X 

X 

Unload  G 

X 

X 

Altitude  Loss 

Seconds  to  First  Throttle 

X 

X 

X 

Seconds  to  First  Roll 

Seconds  to  Recover 

X 

X 

Roll  Input 

X 

X 

Rudder  Input 

Table  12.  Phase  Two  Flight  Testing  Significant  Effect  F  Values  as  Determined  by 
Univariate  Analysis 


Dependent 

Measure 

Upset 

Nose-Low 

Upright 

Nose-High 

Upright 

Nose-Low 

Inverted 

Nose-High 

Inverted 

Altitude 

Loss 

F(l,49)  =  19.48 

p  =  .0001 

F(l,34)  =  5.45 
p  =  .03 

Minimum  Unload 
G  in  Rolls 

Not 

Applicablef 

Maximum  G  Load 
in  Dive  Pullout 

F(l,49)  =  25.52 

p  =  .0001 

F(l,49)  =  10.11 
p  =  .003 

F(l,34)  =  16.02 

p  =  .0001 

F  (1,48)  =  8.912 
p  =  .004 

Seconds  to 
First  Throttle 

F(l,49)  =  14.02 

p  =  .0001 

F(l,34)  =  4.38 
p  =  .04 

F(l,48)  =  7.46 
p  =  .009 

Seconds  to 
First  Roll 

F(l,49)  =  12.19 

p  =  .001 

F(l,49)  =  7.18 

p  =  .01 

F(l,34)  =  17.16 

p  =  .0001 

F(l,48)  =  22.29 

p  =  .0001 

Seconds  to 
Recover 

F(l,49)  =  14.82 

p  =  .0001 

F(l,49)  =  6.83 

p  =  .012 

F(l,48)  =  10.90 

p  =  .002 

fNot  applicable  to  nose-low  upright  upset  because  trained  pilots  were  taught  to  use  rolling  pullouts 
from  dives. 


Table  13.  Comparison  of  Nose-low  Inverted  Upset  Data  with  Unsuccessful  Recoveries 
Included  and  Excluded  (Bolding  =  Significant  Difference) 


Dependent 

Successful  +  Unsuccessful 

Successful  Recoveries  Only 

Ratio-A  / 

Measure 

Trained 

Control 

Ratio-A 

Trained 

Control 

Ratio-S 

Ratio-S 

Altitude 

973.2 

1088.1 

1.118 

948.9 

1069.4 

1.127 

0.992 

Loss 

158.9 

139.9 

167.0 

139.1 

Minimum  Unload 

0.86 

1.33 

1.547 

0.99 

1.41 

1.424 

1.086 

G  in  Rollst 

0.90 

0.60 

0.86 

0.63 

Maximum  G  Load 

4.20 

3.40 

0.810 

4.74 

3.98 

0.840 

0.964 

in  Dive  Pullout 

1.17 

0.99 

0.61 

0.50 

Seconds  to 

3.08 

4.76 

1.545 

2.79 

4.41 

1.581 

0.978 

First  Throttle 

1.96 

2.85 

1.78 

2.81 

Seconds  to 

1.96 

5.24 

2.673 

1.68 

4.88 

2.905 

0.920 

First  Roll 

1.40 

3.29 

0.67 

3.30 

Seconds  to 

7.08 

8.08 

1.141 

7.11 

7.88 

1.108 

1.030 

Recovert 

1.15 

0.95 

1.29 

0.99 

flndicates  that  the  difference  is  significant  when  data  from  unsuccessful  recoveries  are  included  but  loses 
significance  when  data  from  unsuccessful  recoveries  are  eliminated. 
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If  Unload  G  and  Seconds  to  Recover  were  significant  in 
the  nose-low  inverted  upset  recovery,  then  trained  pilots 
would  have  been  statistically  superior  to  untrained  pilots 
in  18  of  23  dependent  measure  categories,  or  78.3%  of 
the  time. 

4.5.  Limitations  of  Ground-Based  Simulator 
Training 

Our  research  results  also  suggest  the  limitations  of 
low-cost  simulator  training  in  developing  upset-recovery 
maneuvering  skills.  MFS  responses  to  control  stick  inputs 
are  realistic  near  the  middle  of  the  Decathlon  flight  enve¬ 
lope  (V-n  diagram).  Near  the  envelope’s  available  G  line 
[aCRIT>  (CL)MAX] ,  however,  responses  to  control  inputs  tend 
to  differ  from  the  Decathlon’s  behavior  in  actual  flight 
situations.  For  example,  if  a  Decathlon  pilot  inadvertently 
stalls  the  airplane  with  an  aileron  down,  a  departure  from 
controlled  flight  may  result.  However,  MFS  does  not 
simulate  departures  from  controlled  flight  realistically. 
MFS  also  responds  inaccurately  during  accelerated  stalls 
in  low-speed  dive  pullouts.  To  recover  the  actual  airplane 
from  such  a  stall,  it  suffices  to  momentarily  relax  back  stick 
pressure  a  small  amount,  resulting  in  a  slightly  reduced  G 
force.  To  recover  the  simulated  Decathlon  from  the  same 
accelerated  stall  situation,  a  pilot  must  unload  completely 
to  0  Gs  for  a  second  or  two.  In  such  scenarios,  the  potential 
for  negative  training  is  significant. 

During  flight  testing,  imperfect  participant  pilot 
control  inputs  frequently  resulted  in  a  nose-high  upset 
progressing  into  low-speed  nose-low  upset.  Whenever 
this  occurred,  accelerated  stalls  and  departures  from 
controlled  flight  were  common.  MFS  limitations  may 
explain  why  these  post-stall  departures  occurred  so 
frequently  and  why  there  was  a  significant  difference 
between  trained  and  control  group  pilot  altitude  losses  in 
nose-low  upsets  but  not  in  nose-high  upsets.  Maneuver¬ 
ing  during  high  energy  nose-low  upsets  occurs  near  the 
middle  of  the  Decathlon  flight  envelope,  where  MFS 
control  responses  are  reasonably  accurate.  By  contrast, 
nose-high  and  low-speed  nose-low  maneuvering  occurs 
near  aCRIT  and  (Cr  ) vfAX,  where  MFS  control  responses 
tend  to  be  inaccurate.  Moreover,  required  control  inputs 


when  maneuvering  in  high-speed  nose-low  upsets  differ 
in  degree,  but  not  in  kind,  from  control  inputs  general 
aviation  pilots  routinely  use  in  non-aerobatic  upright 
maneuvering.  During  nose-high  upset  maneuvering,  by 
contrast,  the  proper  control  inputs  differ  radically  from 
what  general  aviation  pilots  typically  are  accustomed 
to.  As  an  example,  efficient  rolling  in  nose-high  upsets 
requires  a  pilot  to  use  large  aileron  and  rudder  inputs 
at  maximum  thrust  while  maintaining  zero  G  at  air¬ 
speeds  approaching  0  mph.  If  any  significant  positive  or 
negative  G  force  is  applied,  a  stall  and  departure  from 
controlled  flight  may  result.  However,  using  elevator  to 
maintain  0  G  while  applying  large  rudder  and  aileron 
inputs  is  something  general  aviation  pilots  never  expe¬ 
rience.  As  a  second  example,  minimizing  altitude  loss 
in  low-speed  dive  recovery  by  pulling  out  at  the  stall 
buffet  (i.e.,  at  or  very  slightly  above  0lCRIT)  is  again  an 
experience  unfamiliar  to  general  aviation  pilots.  Train¬ 
ing  in  a  flight  simulator  that  responds  inaccurately  to 
control  inputs  in  these  and  similar  situations  makes  it 
difficult  to  prepare  pilots  to  handle  them  adequately  in 
a  real  airplane.  It  is  conceivable  that  the  pilot  behaviors 
necessary  to  perform  nose-high  low  airspeed  precision 
aerobatic  maneuvers  can  be  rehearsed  using  MFS  and 
similar  low-cost  flight  simulators  but  can  only  be  per¬ 
fected  in  a  real  airplane. 

Moreover,  even  when  trained  pilots  significantly 
bettered  control  group  pilots  in  altitude  loss,  their 
performance  was  far  from  optimal.  For  each  of  the  four 
upsets,  Table  14  presents  Phase  Two  average  altitude 
losses  for  trained  and  control  group  pilots.  The  bottom 
row  of  Table  14  reflects  the  minimum  altitude  losses 
that  we  observed  for  each  upset  during  safety  pilot 
training.  27There  is  a  large  disparity  between  research 
participant  altitude  losses  and  the  far  smaller  altitude 
losses  achievable  by  experienced  pilots.  Low-cost  flight 
simulator  training  clearly  improves  a  pilot’s  ability  to 
recover  an  airplane  from  a  serious  upset.  Just  as  clearly, 
however,  it  is  prelude  and  complement  to,  not  a  substi¬ 
tute  for,  aerobatic  experience  in  a  real  airplane.  Whether 
a  better  flight  simulator  would  produce  a  different  result 
during  flight-testing  is  an  open  question. 


Table  14.  Altitude  Losses  to  Nearest  Foot  for  the  Four  Upsets  (Bolding  =  Significant  Effect) 


Data  Source 

Altitude  Loss  in  Feet 

Nose-Low 

Upright 

Nose-High 

Upright 

Nose-Low 

Inverted 

Nose-High 

Inverted 

Trained  Pilot  Average 

565 

331 

949 

382 

Control  Group  Pilot  Average 

728 

340 

1069 

465 

Observed  Minimums  during  Safety  Pilot  Training 

220 

-50 

350 

-30 
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5.  Implications  for  Transport  Type 
Airplanes28 

Phase  Two  flight-test  data  reflect  only  a  modest  dif¬ 
ference  in  performance  between  trained  and  untrained 
participants.  It  may  well  be  the  case  that  simulator-trained 
pilots  need  all-attitude  flight  experience  in  a  real  airplane 
to  hone  simulator-taught  upset-recovery  skills  to  an 
acceptably  high  level.  It  appears  that  during  an  initial 
upset-recovery  experience,  low-cost  simulator  training 
can  help  pilots  overcome  General  Aviation  Syndrome 
only  to  a  limited  extent.  Simulator  shortcomings — for 
example,  unrealistic  control  feedback,  inaccurate  accel¬ 
erated  stall  responses,  and  the  inability  to  replicate  the 
positive  and  negative  G  forces  that  characterize  all-attitude 
flight — limit  a  trainer’s  ability  to  prepare  a  pilot  mentally 
and  emotionally  for  a  real-world  upset.  As  a  result,  any 
subsequent  initial  experience  in  a  real  upsets  may  appear 
strange  and  disquieting.  In  such  a  circumstance,  a  pilot 
easily  loses  situational  awareness  and  instinctively  resorts 
to  old  control  input  habits.  Long-reinforced  patterns  of 
behavior  and  the  significant  stress  of  a  serious  upset  tend 
to  inhibit  the  application  of  new  and  relatively  unfamiliar 
piloting  skills  developed  during  simulator  training. 

Our  research  findings  seem  to  call  in  question  the 
implicit  assumption  that  airline  simulator-based  upset- 
recovery  training  programs  impart  flying  skills  sufficient 
to  make  it  probable  that  typical  line  pilots  can  recover 
an  airliner  from  a  serious  upset.  It  is  true  that  airline 
pilots,  on  average,  are  considerably  more  experienced 
than  our  research  participants,  hence  may  benefit  more 
from  upset-recovery  training.  However,  air  transport 
pilots'  experience  consists  of  hours  of  flying  straight  and 
level,  punctuated  by  occasional  excursions  into  very  small 
bank  and  pitch  angles.  In  addition,  airline  pilots  typi¬ 
cally  receive  only  about  four  hours  of  classroom-based 
upset-recovery  training  and  perhaps  an  hour  of  simula¬ 
tor  training,  in  comparison  to  ten  hours  of  each  for  our 
trained  participants.  Moreover,  the  primary  advantage  of  a 
Level-D  simulator  over  low-cost  desktop  flight  simulation 
is  limited  to  cockpit  verisimilitude  and  realistic  control 
forces.  Typically,  the  motion  of  a  Level-D  simulator  is 
disabled  to  avoid  stressing  the  mechanism  unnecessarily  in 
upset-recovery  maneuver.  In  any  event,  Level-D  simulator 
motion  is  in  no  way  realistic  in  all-attitude  maneuvering, 
and  neither  Level-D  nor  low-cost  desktop  simulators  can 
replicate  the  G  forces  that  characterize  upset-recovery 
maneuvering  in  a  real  airplane. 

Thirtyyears  ago,  U.S.  airline  pilots  typically  came  from 
military  flight  backgrounds  where  training  afforded  them 
extensive  opportunity  to  perform  aerobatic  flight  maneu¬ 
vers  .  These  pilots  were  routinely  accustomed  to  all-attitude 


flight.  For  them,  there  were  no  unusual  attitudes,  only 
unexpected  attitudes.  By  contrast,  most  air  transport  pilots 
flying  today  lack  a  military  background  and  have  never 
experienced  the  extreme  pitch  and  bank  angles  and  high 
G  forces  associated  with  severe  airplane  upsets.  Indeed, 
most  have  never  even  been  upside-down  in  an  airplane. 
Informal  conversations  with  current  airline  pilots  suggest 
that  while  virtually  all  regard  the  upset  training  they  receive 
as  useful,  a  significant  number  also  perceive  it  as  a  pro 
forma  approach  to  a  serious  safety  problem — better  than 
nothing  but  far  from  what  would  be  desirable  if  training 
costs  were  not  a  paramount  consideration.  In  short,  it 
seems  unlikely  that  airline  upset  training  is  a  completely 
acceptable  substitute  for  all-altitude  manuevering  experi¬ 
ence  in  a  real  airplane. 

Upsets  are  known  to  be  a  primary  cause  of  fatal  com¬ 
mercial  air  transport  accidents.  Passenger  and  air  crew 
safety  considerations  mandate  that  air  transport  pilots 
be  able  to  recover  from  the  infrequent  but  potentially 
catastrophic  upsets,  which  inevitably  will  occur  from 
time  to  time  in  air  transport  operations.  Our  research 
implies  that  aerobatic  experience  in  a  real  airplane  may 
be  required  to  make  recovery  probable  with  minimum 
altitude  loss.  Since  aerobatic  experience  cannot  be  ob¬ 
tained  legally  in  transport  type  aircraft,  perhaps  the  FAA 
should  consider  making  aerobatic  experience  in  a  light 
airplane  part  of  the  requirement  for  a  commercial  pilot 
license  and/or  an  ATP  rating. 

6.  Follow-On  Research 

In  August  2008,  we  commenced  follow-on  upset-re¬ 
covery  research  in  partnership  with  Dick  Leland  of  ETC, 
lead  researcher  and  financial  sponsor  of  the  project  under 
an  FAA  grant.  Twenty-five  Embry- Riddle  flight  students 
received  the  same  ten  hours  of  classroom  training  that 
Phase  Two  trained-participants  received.  They  received 
five  hours  ofMFS  training,  then  traveled  in  small  groups 
to  Pennsylvania  to  receive  five  hours  of  weekend  upset- 
recovery  training  in  ETC’s  proprietary  centrifuge-based 
GyroLab-2000  flight  simulator.  ETC  modified  the 
GyroLab-2000  to  give  it  the  flight  characteristics  of  a 
Super  Decathlon  airplane  and  to  make  its  basic  flight 
instruments  replicate  the  layout  on  a  Decathlon’s  instru¬ 
ment  panel. 

When  their  Gyro  Lab  training  was  complete,  in  early 
November  2008,  the  25  follow-on  research  participants 
underwent  Decathlon  flight  testing  identical  to  that 
received  by  Phase  Two  participants.  Their  performance 
in  recovering  from  the  four  upsets  will  be  compared  with 
the  performance  of  Phase  Two-trained  and  control  group 
pilots.  The  follow-on  research  is  designed  to  determine 
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Table  15.  The  3x4  Repeated  Measures  Factorial  Design  of  the  Follow-On  Research  Experiment 


3x4  Factorial 

Upset  Attitude  (Repeated  Measure) 

Nose-high 

Upright 

Nose-low 

Upright 

Nose-high 

Inverted 

Nose-low 

Inverted 

Training 

10  Flours  Classroom  /  5  Flours 
MFS  +5  Hours  GyroLab-2000 

GyroLab/MFS 
Trained  Pilots 

GyroLab/MFS 
Trained  Pilots 

GyroLab/MFS 
Trained  Pilots 

GyroLab/MFS 
Trained  Pilots 

10  Hours  Classroom  /  10  Hours 
MFS  Training 

MFS  Trained 
Pilots 

MFS  Trained 
Pilots 

MFS  Trained 
Pilots 

MFS  Trained 
Pilots 

None 

Control  Group  Pilots 

Untrained  Pilots 

Untrained  Pilots 

Untrained  Pilots 

Untrained  Pilots 

the  added  value  of  upset  training  in  a  motion-based  flight 
simulator  capable  of  generating  continuous  G  forces.  We 
hypothesize  that  pilots  trained  in  the  sophisticated  Gyro- 
Lab-2000  simulator  will  outperform  Phase  Two-trained 
and  untrained  pilots  in  upset- recovery  maneuvering.  Table 
15  reflects  the  fact  that  the  experimental  design  of  the 
follow-on  research  is  a  3x4  repeated  measures  factorial. 
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